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Summary 

* 

\.  1 

Three  major  objectives  were  completed  during  the  year. 

The  first  demonstrates  how  to  directly  use  rank-one  updates 
to  a  Cholesky  factorization  of  the  required  inverse  for 
Karmarkar  projections  while  fully  exploiting  sparsity.  This 
can  significantly  improve  computational  speed  when  only  a 
few  variables  are  changing  significantly  at  each  step.  The 
second  demonstrates  a  new  method  for  adding  new  variables  to 
a  quasi-Newton  Hessian  approximation  which  preserves  problem 
scale  and  positive  definiteness  of  the  Hessian.  Numerical 
results  show  the  method  to  be  preferable  to  known  methods. 

The  third  examines  a  variety  of  ways  of  implementing  a 
sequential  quadratic  programming  code,  and  uses  numerical 
testing  to  indicate  a  suitable  merit  function  and  good 
algorithms  for  updating  Lagrange  multiplier  and  Hessian 
approximations.  Recent  new  results  for  updating  Hessians 
for  unconstrained  problems  are  currently  being  studied  to 

determine  if  better  Hessian  approximations  can  be  obtained.i *ror 
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The  proposed  research  was  to  investigate  numerical 
methods  for  two  constrained  optimization  problems,  linear 
programming  problems  and  nonlinear  programming  problems  with 
either  linear  or  nonlinear  equality  constraints.  This 
report  will  deal  with  progress  to  date  and  remaining 
problems  of  interest  for  each  of  these  in  turn. 


The  general  nonlinear  programming  problem  is 


minimize 


f(x) 


subject  to  g^(x)  =0,  i*l,...,m 

and  9j(x)<0,  j=m+l, . . . ,p. 


Here  x»  (xlf...,xn)  ,  and  f(x)  is  assumed  to  be  nonlinear  in 
at  least  a  subset  of  the  variables.  When  the  constraints 
are  linear,  it  is  well  known  that  this  problem  can  be 
transformed  to  the  form 


minimize  f(x) 
subject  to  Ax=b, 
lSx<u, 


where  1  and  u  are  vectors  of  (possibly  infinite)  upper  and 
lower  bounds.  Reduced  gradient  methods  fix  some  of  the 
variables  at  their  bounds,  and  solve  the  reduced  problem  in 
the  remaining  free  variables.  The  fixed  variables  are  then 
tested  to  ascertain  whether  a  better  solution  can  be  found 
if  the  variable  is  allowed  to  vary  freely  between  its 
bounds.  When  a  variable  is  freed,  computation  of  a  search 
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direction  requires  that  either  the  mixed  second  partial 
derivatives  of  f(x)  with  respect  to  the  variable  are 
explicitly  calculated,  or  if  a  quasi-Newton  approximation  to 
the  Hessian  is  being  used,  that  they  be  estimated  in  a  way 
consistent  with  problem  scaling  and  maintaining  positive 
definiteness  of  the  approximate  Hessian.  Finite 
differencing  will  produce  an  approximation  which  is 
appropriately  scaled,  but  the  approximate  Hessian  need  not 
be  positive  definite.  Adding  a  row  and  column  with  a 
positive  diagonal  and  zeroes  elsewhere  will  produce  a 
positive  definite  Hessian,  but  it  is  generally  poorly 
scaled. 

This  research  has  devised  a  new  method,  identical  in 
the  amount  of  work  to  finite  differencing,  but  which  uses 
the  structure  of  quasi-Newton  methods  to  assure  both 
positive  definiteness  and  proper  scaling.  The  method  has 
been  tested  numerically,  and  in  limited  testing  has  proved 
far  superior  to  current  alternative  methods.  It  has  been 
documented  in  the  report  "Adding  Variables  to  Quasi-Newton 
Hessian  Approximations",  which  will  appear  in  the  Journal  of 
Optimization  Theory  and  Applications  in  August,  1987. 

When  the  constraints  are  nonlinear,  transformation  to 
slacked  equality  constraints  may  or  may  not  cause  numerical 
problems  with  singular  Hessian  matrices.  When  only  the 
Hessian  of  f(x)  is  considered,  clearly  it  is  singular  if 
slack  variables  are  included  in  the  constraints.  If, 
however,  the  Hessian  is  the  Hessian  of  the  Lagrangian 
function,  then  singularity  may  not  occur.  While  the 
problem  of  transforming  inequality  to  equality  constraints 
for  general  nonlinear  programming  is  still  the  subject  of  a 


great  deal  of  research ,  enough  problems  can  be  formulated 
with  strict  equality  constraints  to  generate  substantial 
interest  in  solving  the  problem 

minimize  f(x) 

subject  to  g^(x)  "0,  i«l,...,m. 

The  Langrangian  of  this  problem  is 

L(x,  )«  f(x)-£>igi(x)/ 

and  the  currently  most  successful  methods  for  solving  the 
problem  are  the  sequential  quadratic  programming  methods, 
which  attempt  to  solve  the  problem  by  computing  the  first 
order  conditions  for  a  solution  by  differentiating  the 
Lagrangian,  and  then  applying  Newton's  method  to  solve  these 
first  order  conditions.  Here  the  search  directions  can  be 
shown  to  solve 

T  T 

minimize  s  Bs+s  Vf 

T 

subject  to  g+N  s*0, 

where  N  is  the  matrix  of  constraint  gradients, Vf  the 
gradient  of  the  objective  function,  g=(g1, . . . ,g  )  ,  and  B  is 
an  approximation  to  the  Hessian  of  the  Lagrangian. 

As  B  is  an  estimate  to  the  Hessian  of  the  Lagrangian, 
calculation  of  B  requires  estimates  to  the  Lagrange 
multipliers.  Further,  a  sufficient  step  along  s  to  assure  a 
quasi-Newton  approximation  to  B  is  positive  definite  can  not 


always  b«  taken,  for  the  new  point  may  prove  to  be  too  far 
outside  the  feasible  region.  Finally,  some  measure,  called 
a  merit  function,  must  be  Introduced  to  assure  that 
feasibility  and  function  reduction  are  kept  In  reasonable 
balance. 

The  goal  of  this  research  was  to  attempt  to  test 
various  means  of  estimating  ^  and  B  ,  and  to  use  several 
different  merit  functions  in  an  attempt  to  determine  what 
combination  was  likely  to  produce  an  efficient  numerical 
algorithm.  The  results  of  this  study  have  been  documented 
in  the  report  "Numerical  Experience  wiht  Sequential 
Quadratic  Programming",  which  is  under  review  for  the 
Transactions  on  Mathematical  Software  in  revised  form, 
having  been  revised  to  address  the  comments  of  the  generally 
favorable  referees.  The  report  addresses  the  questions 
raised  above  in  detail,  with  specific  recommendations  on 
merit  functions,  update  strategy,  and  Lagrange  multiplier 
estimates. 


Finally,  research  is  being  concluded  on  more  efficient 
hybrid  schemes  for  updating  a  Hessian  approximations,  and 
will  be  documented  in  a  report  to  be  submitted  to  the  SIAM 


Journal  on  Numerical  Analysis. 


Recent  research  in  linear  programming  has  been 
concerned  with  interior  point  methods,  or  methods  which 
follow  a  trajectory  across  the  interior  of  the  feasible 
region  rather  than  moving  from  vertex  to  vertex.  This  was 
initiated  by  Karmarkar,  who  used  projective  transformations 
to  recenter  the  current  estimate  at  every  point,  then  move 


toward  tha  boundary. 


The  major  numerical  work  in  Karmarkar* s  algorithm  is  in 
computing  a  projection  of  the  form 

s  -  (I-DAT(AD2  AT)-1AD)c, 

where  D  is  a  diagonal  matrix  which  changes  at  each 
iteration,  while  A  is  a  fixed  matrix.  Karmarkar  suggests 
updating  a  factorization  of  (AD  A  )  only  for  those  elements 
of  D  which  have  changed  significantly,  and  shows  that  this 
reduces  the  theoretical  complexity  of  the  algorithm. 

The  report  "Computing  Karmarkar  Projections  Quickly", 

has  been  accepted  for  publication  in  Mathematical 

Programming.  It  shows  how  to  update  a  Cholesky 

2  T 

factorization  to  AD  A  in  a  way  that  fully  and  naturally 
exploits  sparsity,  and  shows  that  the  method  has  great 
promise  for  reducing  computation  time. 

Finally,  significant  research  is  continuing  on  interior 
point  methods.  A  code  implemeting  a  full  primal-dual  method 
is  under  development,  with  very  promising  preliminary 
results.  This  is  being  continued  under  grant  AFSOR-87-0215 
to  Rutgers  University. 
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Articles  Accepted,  Submitted,  and  Planned 

1)  D.  F.  Shanno  and  K.  H.  Phua,  "Adding  Variables  to 
Quasi-Newton  Approximations",  Journal  of  Optimization 
Theory  and  Application  (in  press). 

2)  D.  F.  Shanno  and  K.  H.  Phua,  "Numerical  Experience  with 
Sequential  Quadratic  Programming  Algorithms  for  Equaltity 
Constrained  Nonlinear  Programming",  Transactions  on 
Mathematical  Software  (under  review) . 

3)  D.  F.  Shanno,  "Computing  Karmarkar  Projections  Quickly", 
Mathematical  Programming  (in  press). 

4)  D.  F.  Shanno,  "Improved  Quasi-Newton  Approximations", 

(in  preparation) . 

Paper  presented  at  meetings 

Computing  Karmarkar  Projections  Quickly  at  the  SIAM  National 

Meeting,  Boston,  MA. ,  July,  1986  and  at  the  ORSA/TIMS 

Meeting,  Miami,  October,  1986  (both  invited) . 
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